Skip to content

quic: apply multiple quic performance and packet handling improvements#63267

Open
jasnell wants to merge 28 commits into
nodejs:mainfrom
jasnell:jasnell/quic-backend-improvements
Open

quic: apply multiple quic performance and packet handling improvements#63267
jasnell wants to merge 28 commits into
nodejs:mainfrom
jasnell:jasnell/quic-backend-improvements

Conversation

@jasnell
Copy link
Copy Markdown
Member

@jasnell jasnell commented May 12, 2026

A number of planned backend changes and improvements in the packet handling. Yields a net performance improvement of around 10% higher rps. Sets us up to be able to better leverage libuv/libuv#5116 if/when that lands (@santigimeno @saghul fyi) which will provide an even larger perf boost. Fixes a bug while we're at it.

@nodejs/quic

@nodejs-github-bot
Copy link
Copy Markdown
Collaborator

nodejs-github-bot commented May 12, 2026

Review requested:

  • @nodejs/quic

@jasnell jasnell requested review from Qard and pimterry May 12, 2026 18:52
@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels May 12, 2026
@jasnell jasnell changed the title meta: add additional gitignore entries quic: apply multiple quic performance and packet handling improvements May 12, 2026
@nodejs-github-bot

This comment was marked as outdated.

Sign in to view
@codecov
Copy link
Copy Markdown

codecov Bot commented May 12, 2026 •
edited
Loading

Codecov Report

❌ Patch coverage is 99.55605% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.07%. Comparing base (2d6cbea) to head (25af788).
⚠️ Report is 7 commits behind head on main.

Files with missing lines Patch % Lines
lib/internal/fs/promises.js 66.66% 2 Missing ⚠️
src/dataqueue/queue.cc 60.00% 0 Missing and 2 partials ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main   #63267      +/-   ##
==========================================
+ Coverage   90.06%   90.07%   +0.01%     
==========================================
  Files         714      714              
  Lines      225502   226016     +514     
  Branches    42628    42646      +18     
==========================================
+ Hits       203089   203581     +492     
- Misses      14195    14200       +5     
- Partials     8218     8235      +17
Files with missing lines Coverage Δ
lib/internal/quic/quic.js 100.00% <ø> (ø)
lib/internal/quic/state.js 100.00% <100.00%> (ø)
lib/internal/quic/stats.js 100.00% <100.00%> (ø)
lib/internal/quic/symbols.js 100.00% <ø> (ø)
src/aliased_struct-inl.h 90.90% <ø> (ø)
src/aliased_struct.h 100.00% <ø> (ø)
lib/internal/fs/promises.js 92.87% <66.66%> (-0.08%) ⬇️
src/dataqueue/queue.cc 67.94% <60.00%> (+0.31%) ⬆️

... and 28 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@nodejs-github-bot

This comment was marked as outdated.

Sign in to view
@jasnell jasnell force-pushed the jasnell/quic-backend-improvements branch from e921c66 to ef9e8da Compare May 13, 2026 03:54
@nodejs-github-bot

This comment was marked as outdated.

Sign in to view
@nodejs-github-bot

This comment was marked as outdated.

Sign in to view
@jasnell jasnell requested a review from mcollina May 14, 2026 13:37
@jasnell jasnell added the experimental Issues and PRs related to experimental features. label May 14, 2026
@jasnell jasnell force-pushed the jasnell/quic-backend-improvements branch 4 times, most recently from bbe613a to 6ef4e68 Compare May 15, 2026 13:41
@Ethan-Arrowood
Copy link
Copy Markdown
Contributor

Ethan-Arrowood commented May 15, 2026

This diff is quite large. Can you provide a short summary of the most relevant parts? Such as maybe notable performance improvement areas, the bug fix, and whatever else seems relevant. That'll help expedite reviews, thanks!

@jasnell jasnell force-pushed the jasnell/quic-backend-improvements branch 2 times, most recently from bfc3adc to c9c714e Compare May 15, 2026 16:56
@jasnell jasnell mentioned this pull request May 15, 2026
Open
@jasnell
Copy link
Copy Markdown
Member Author

jasnell commented May 15, 2026 •
edited
Loading

Just an idea of where we're at... running a local 'hello world' benchmark using h2load as the client driver on the same 88 core machine... 4 runs with 100,000 requests over 100 concurrent connections:

image

The rps is not as high as I'd like and http/1 still beats it by a long way but it's a solid result for current state.

The primary performance bottleneck right now is the JS side, GC, and the C++/JS boundary cross. We lose a fair amount of throughput in the context switching.

My current goal is to get the per-request down to 10 microseconds.

@jasnell
Copy link
Copy Markdown
Member Author

jasnell commented May 15, 2026 •
edited
Loading

Ok, with a bit of perf tunning and mangling with both h2load options and v8 perf tuning options on the server, with a node binary built with pointer compression enabled... we can get ~101k-102k rps. Still slower than http/1, on par with http/2 when pointer compression is enabled. Overall a great result given that I've only just started the perf tuning of the implementation.

/tmp/opencode/local/bin/h2load --h3 -n 200000 -c 5 -w 26 -W 30 -m 10 -D 2 https://127.0.0.1:4433
starting benchmark...
spawning thread #0: 5 total client(s). Timing-based test with 0s of warm-up time and 2s of main duration for measurements.
Warm-up started for thread #0.
progress: 20% of clients started
progress: 40% of clients started
progress: 60% of clients started
progress: 80% of clients started
progress: 100% of clients started
Warm-up phase is over for thread #0.
Main benchmark duration is started for thread #0.
TLS Protocol: TLSv1.3
Cipher: TLS_AES_128_GCM_SHA256
Certificate: ECDSA P-256 256 bits
Negotiated Group: X25519
Resumption: no
Application protocol: h3
Main benchmark duration is over for thread #0. Stopping all clients.
Stopped all clients for thread #0

finished in 2.02s, 102130.00 req/s, 2.23MB/s
requests: 204260 total, 204310 started, 204260 done, 204260 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 204260 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 4.46MB (4676960) total, 797.97KB (817120) headers (space savings 87.50%), 2.14MB (2247080) data
UDP datagram: 33273 sent, 20462 received
                 min         max         median      p95         p99         mean        sd         +/- sd
request     :      150us     53.98ms       435us       490us       813us       487us      1.28ms    99.59%
connect     :     4.89ms      5.64ms      5.30ms      5.64ms      5.64ms      5.28ms       295us    60.00%
TTFB        :     8.11ms      8.25ms      8.18ms      8.25ms      8.25ms      8.18ms        54us    60.00%
req/s       :   20402.15    20442.34    20417.92    20442.34    20442.34    20421.97       17.40    60.00%
min RTT     :       22us       115us        38us       115us       115us        57us        40us    80.00%
smoothed RTT:       86us       226us       164us       226us       226us       163us        60us    60.00%
packets sent:       6219        7436        6373        7436        7436     6656.60      545.25    80.00%
packets recv:       4092        4099        4092        4099        4099     4094.40        3.36    80.00%
packets lost:          0           0           0           0           0        0.00        0.00   100.00%
GRO packets :          1           1           1           1           1        1.00        0.00   100.00%

jasnell added 5 commits May 15, 2026 16:08
@jasnell
meta: add additional gitignore entries
22c2e34 
Signed-off-by: James M Snell <jasnell@gmail.com>
@jasnell
quic: improve backend quic packet processing
cabae1e 
Use a uv_check_t on BindingData to process outbound
pending packet send, and use TrySend for actually sending
packets when possible. Results in an 8% improvement in
req/s and ~24% improvement in p95 latency.

Also sets us up better for future improvements in
libuv if the changes proposed in libuv/libuv#5116
are accepted.

Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode:Opus 4.6
@jasnell
test: fixup quic tests
5360171 
Signed-off-by: James M Snell <jasnell@gmail.com>
@jasnell
quic: improve batching of packet sending
6011429 
Improves overall performance and sets us up for eventual
support for GRO/GSO batching in libuv in the future.

Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode:Opus 4.6
@jasnell
quic: add support for future ECN marking
3e09dd6 
Set up for when libuv eventually supports ECN marking.
Pass the ECN marking stuff into ngtcp2.

Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode:Opus 4.6
jasnell added 16 commits May 15, 2026 16:08
@jasnell
quic: cache the timestamp on send and receive
8afc8d0 
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode:Opus 4.6
@jasnell
quic: eliminate per-received datagram allocation
22fa893 
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode:Opus 4.6
@jasnell
quic: fix crash in early handshake failure
af6e9e1 
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode:Opus 4.6
@jasnell
quic: fixup linting issue after other changes
5107283 
Signed-off-by: James M Snell <jasnell@gmail.com>
@jasnell
quic: implement rate limiting for version nego and immediate close
5a73c7d 
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode:Opus 4.6
@jasnell
quic: add handshake timeout and default connection limits
461fece 
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode:Opus 4.6
@jasnell
quic: add aliased struct arenas
fea579f 
Every stream, session, and endpoint creates aliased
structs for stats and state. These were creating
v8::ArrayBuffer allocations and views for each instance,
which is expensive. This adds a new arena mechanism
for AliasedStructs that allocates in pages and allows
Streams and Sessions to share the same underlying
ArrayBuffer for their stats and state. Since these
are never exposed to users, this is safe and results
in a significant reduction in allocation counts.

Each arena maintains a freelist of pages, where each
page is a max of 16KB bytes. Pages are lazily allocated
and freed as needed. Each slot in the arena corresponds
to a single struct instance, and the slot index is
used to calculate the byte offset within the page for
that struct's view.

The perf improvement is modest but measurable. The
key benefit is in reduced memory fragmentation and GC
overhead.

Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode:Opus 4.6
@jasnell
quic: improve internal structure of QuicStream
7b3d437 
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode:Opus 4.6
@jasnell
quic: improve the quic js structure
4349b5a 
* Adds invalid this protections in the stats classes
* Collects internal private fields into a single private symbol
  to reduce the number of private field accesses
* Adds and corrects isQuic* class checks
* Preserve stream id and direction after destroy
* Improve stats close snapshotting
* Fixes a handful of other bugs and doc issues

Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode:Opus 4.6
@jasnell
quic: fix UAF in Application::OnTimeout()
19dc615 
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode:Opus 4.6
@jasnell
quic: fixup UAFs in bindingdata, streams, and app
cd5f352 
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode:Opus 4.6
@jasnell
quic: fix premature unref of endpoint when listening
159c0bc 
Signed-off-by: James M Snell <jasnell@gmail.com>
@jasnell
quic: fixup some v8:: qualifiers
6c2ba1d 
Signed-off-by: James M Snell <jasnell@gmail.com>
@jasnell
quic: fix tests that are missing serverEndpoint close
15f22b4 
Signed-off-by: James M Snell <jasnell@gmail.com>
@jasnell
quic: apply multiple additional minor improvements
c0551c3 
Signed-off-by: James M Snell <jasnell@gmail.com>
@jasnell
quic: coalesce received data into fewer buffers
fdef9eb 
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode:Opus 4.6
@jasnell jasnell force-pushed the jasnell/quic-backend-improvements branch from c9c714e to fdef9eb Compare May 15, 2026 23:09
@jasnell
quic: add reusePort option to QuicEndpoint
c429d16 
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode:Opus 4.6
@jasnell jasnell force-pushed the jasnell/quic-backend-improvements branch from 7c159ef to c429d16 Compare May 16, 2026 02:52
jasnell added 2 commits May 15, 2026 21:48
@jasnell
quic: improve stream header collection performance
bd2e04d 
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode:Opus 4.6
@jasnell
quic: enable recvmmsg batching in Endpoint
46a16bb 
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode:Opus 4.6
martenrichter
martenrichter reviewed May 16, 2026
View reviewed changes
Comment thread test/parallel/test-quic-stream-recv-coalescing.mjs
@jasnell
quic: add initial RTT option to session options
9651a8f 
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: OpenCode:Opus 4.6
@jasnell
Copy link
Copy Markdown
Member Author

jasnell commented May 16, 2026

Quick status update on this PR: this makes a range of improvements throughout the codebase representing some intensive benchmarking and testing I've done this week. On my local machine, I've been able to achieve over 2 million http/3 rps with 32 parallel server processes (reusePort: true) with no packet loss, no memory leaks, no failures. The remaining bottlenecks are pretty much entirely on the JS side or involve the JS/C++ boundary crosses.

I've got a libuv PR (libuv/libuv#5116) that introduces ECN, MTU path discovery, GSO/GRO support, and a few other bits that will yield further improvements (tho GSO/GRO are likely the only ones that will impact local benchmark results).

This is a big PR, yes, but waiting for the Small PR, Review, Land, Iterate on this is painfully slow and would likely draw this out for months more. Hopefully we're nearing the end of needing to do these larger batches.

jasnell added 3 commits May 16, 2026 06:51
@jasnell
quic: improve recv coalescing test sizes
25af788 
Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode:Opus 4.6
@jasnell
quic: add getters for local and remote transport parameters
1cded9d 
Signed-off-by: James M Snell <jasnell@gmail.com>
@jasnell
quic: add applicationOptions to session
4981fc5 
Signed-off-by: James M Snell <jasnell@gmail.com>
@jasnell jasnell force-pushed the jasnell/quic-backend-improvements branch from 42cd7a0 to 4981fc5 Compare May 16, 2026 16:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Qard Qard Awaiting requested review from Qard

@pimterry pimterry Awaiting requested review from pimterry

@mcollina mcollina Awaiting requested review from mcollina

1 more reviewer

@martenrichter martenrichter martenrichter left review comments

Reviewers whose approvals may not affect merge requirements

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

c++ Issues and PRs that require attention from people who are familiar with C++. experimental Issues and PRs related to experimental features. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@jasnell @nodejs-github-bot @Ethan-Arrowood @martenrichter